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Although afterschool programs for children have been operating for decades in some 
communities, the afterschool movement — the great national awakening to the opportunity 
afterschool offers — is just a few years old. As public demand for afterschool has grown, so has 
the demand for accountability. That is particularly true in afterschool programs that spend public 
dollars. After all, where tax dollars flow, so must accountability to taxpayers. 

Fortunately for afterschool advocates, a steady stream of afterschool evaluations are 
documenting gains for children, especially those who regularly participate in afterschool 
programs and those at highest risk of academic failure. This updated evaluations backgrounder 
includes summaries of: 

> a meta-analysis covering 35 studies, by Mid-continent Research for Education and Learning 
(McREL); 

> a meta-analysis covering 73 studies, by the Collaborative for Academic, Social, and 
Emotional Learning (CASEL); 

> a study of 35 high-quality afterschool programs in California, Colorado, Connecticut, 
Michigan, Montana, New York, Oregon and Rhode Island, by researchers at the University 
of California, Irvine, the University of Wisconsin-Madison and Policy Studies Associates, 
Inc.; 

> a new study of Chicago’s After School Matters, by researchers at the Chapin Hall Center for 
Children at the University of Chicago; 

> a study of academically focused New Hampshire programs by the RMC Research 
Corporation; and 

> a new study of the five-city, California-based Communities Organizing Resources to 
Advance Learning (CORAL), conducted by Public/Private Ventures. 

Summaries of other evaluations cover studies of LA’s BEST, Citizen Schools, the YMCA of 
Greater New York’s Virtual Y Program, the Young Scholars Program, Generacion Diez, 21 st 
Century Community Learning Centers throughout Texas, the Massachusetts After-School 
Research Study, The After-School Corporation (TASC), Loundations Inc., Project Learn, San 
Diego’s ‘6 to 6’ and more. A list of the studies and their key findings is provided at the end of 
this document on Page 3 1 . 
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This compilation focuses chiefly on evaluations of afterschool programs showing positive gains 
on student academic achievement. A second backgrounder, available from the Afterschool 
Alliance website at afterschoolalliance.org , summarizes findings related to student safety, 
behavior, substance-abuse-prevention and discipline. 



The Landscape of Afterschool Evaluations 

A number of different types of evaluations have been conducted over the last several years, 
assessing various aspects of afterschool programming. Some evaluations seek to gather data on 
whether programs have been structured as they were originally intended, how well they have 
done at meeting attendance and staffing goals, how they “fit” in the school environment and 
more. Others explore student and other outcomes — the effects afterschool programs have on the 
children who participate in them, their parents and even the communities at large. 

Both types of evaluations are of great value to afterschool providers and to policy makers, and 
when taken together the two types of studies help identify the particular program elements and 
approaches most critical to accomplishing program goals. For example, studies correlating 
increased afterschool attendance with increased academic performance have triggered 
considerable discussion in the afterschool community about ways to improve student attendance 
in afterschool programs. 

Evaluations also differ by virtue of who conducts them. Many programs self-evaluate, providing 
useful data and satisfying the needs of their various stakeholders: parents, funders, partnering 
businesses, local public officials and so on. But for academics and large funders — the federal 
government, state governments, the Open Society Institute, the Charles Stewart Mott Foundation 
or the Wallace Fund, for example — more exacting standards and greater independence is often 
required. Such independent evaluations commissioned by such entities are the primary subject 
of this document. 

The studies included in this backgrounder are just a few of the numerous evaluations of 
afterschool programs completed in recent years. To learn more about afterschool evaluation and 
to search a more comprehensive database of evaluations, visit the Harvard Family Research 
Project at http://www.hfrp.org/out-of-school-time. 



Summary Lessons from the Data 

In reviewing the studies included in this backgrounder, a few key themes emerged. Below we 
present the summary lessons from the studies included in this backgrounder. The data and 
conclusions from these studies suggest that quality afterschool programs can have a positive 
impact on a number of measures of student academic achievement. Detailed descriptions and 
findings from all the studies, including citations, are included in the Afterschool Evaluations in 
Detail section beginning on page seven. See Appendix A, Afterschool Evaluations at a Glance 
on page 3 1 for a summary listing of the studies included in this backgrounder. 
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Improved School Attendance and Engagement in Learning 



> Chapin Hall’s study of Chicago’s After School Matters program found that students who 
participated in the program missed fewer days of school than their classmates, and that 
students who participated most frequently failed fewer core academic courses (English, 
Math, Science and Social Studies). The finding is particularly noteworthy because the 
program is not aimed at improving academics. By creating an incentive for students to 
attend school regularly and giving them something to look forward to after school, 
researchers conclude, After School Matters helped improve academic performance. 

> A study conducted for the state of New Hampshire of its state- and federally-funded, 
academically focused afterschool programs found that more than half the students who 
attended regularly improved both academically and behaviorally and classroom teachers 
reported that students made the most progress in turning in homework on time, 
completing homework to teachers’ satisfaction and participating in class. 

> Evaluations of LA’s BEST by the UCLA Center for the Study of Evaluation revealed that 
students’ regular school-day attendance improved once they began participating in the 
afterschool program. Students also reported higher aspirations regarding finishing school 
and going to college. In a longitudinal study, researchers at UCLA found drop out rates 
among LA’s BEST students are 20 percent lower than the overall district drop out rate. 

> Research by the Texas State Education Agency found that strong participation in 21 st 
CCLC afterschool programs correlated with better attendance during the regular school 
day. “Approximately half (48 percent) of youth who participated in three quarters or 
more of the available 21 st CCLC activities missed five or fewer days of school during the 
fall semester, compared to 17 percent of youth who participated in less than one quarter 
of the available activities. 

> Policy Studies Associates five year evaluation of The After-School Corporation (TASC) 
afterschool programs found that regular school day attendance for the pre-K to eighth 
grade levels increased for participants compared to non-participants, especially for 
students in grades 5-8, and attendance for seventh and eighth grade participants increased 
by 2.7 school days in comparison to non-participants. At the high school level, regular 
school day attendance for students in the lowest quartile of attendance increased by 4.4 
days, compared to non-participants who were also in the lowest attendance quartile. 

> Pathways to Progress students in St. Paul, Minnesota experienced dramatically better 
school attendance — participants attended 18.44 more school days and missed 9.57 fewer 
school days than their nonparticipant peers. 

> An evaluation of Boys & Girls Clubs’ Project Learn found that engagement in reading, 
use of verbal skills, writing, tutoring and the study of geography all significantly 
increased as program involvement increased. 

> Students participating in California’s After School Education and Safety Program 
(formerly the After School Learning and Safe Neighborhoods Partnerships Program, 
ASLSNPP) improved their regular day attendance; with particularly large improvements 
for students with the lowest attendance record prior to the start of the program. Students 
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also demonstrated a more positive attitude toward school, enhanced confidence about 
learning and increased educational aspirations. 

> Absences among Young Scholars participants decreased by 48 percent when they began 
participating in the afterschool program. 

> A five-site evaluation of the Boys & Girls Clubs’ national Project Learn program found a 
reduction in absences among participants, from 6.4 days per school year at baseline to 
2.19 days per school year at follow-up. This was especially notable when compared to 
non-participants whose absences increased over that same 30 month period. 

Improved Test Scores and Grades 

> The Promising Afterschool Programs Study, a study of about 3,000 low-income, 
ethnically-diverse elementary and middle school students, found that those who regularly 
attended high-quality programs over two years demonstrated gains of up to 20 percentiles 
and 12 percentiles in standardized math test scores respectively, compared to their peers 
who were routinely unsupervised during the afterschool hours. 

> A meta-analysis by the University of Illinois at Chicago-based Collaborative for 
Academic, Social, and Emotional Learning (CASEL), “The Impact of After-School 
Programs that Promote Personal and Social Skills,” found that “Youth who participate in 
after-school programs improve significantly in three major areas: feelings and attitudes, 
indicators of behavioral adjustment, and school performance. More specifically, after- 
school programs succeeded in improving youths’ feelings of self-confidence and self- 
esteem, school bonding (positive feelings and attitudes toward school), positive social 
behaviors, school grades and achievement test scores.” The meta-analysis spanned 73 
separate studies of afterschool programs. 

> Students participating in LA’s BEST afterschool programs demonstrated higher academic 
achievement on standardized tests of math, reading and language arts. In addition, 
language redesignation rates favored LA’s BEST students when compared with non-LA’s 
BEST students. 

> Prior to participating in Young Scholars, many of the participants were retained or 
required to repeat a grade each year. Over five years, promotion rates for Young 
Scholars improved by 83 percent. Young Scholars participants also made significant 
gains on North Carolina’s state tests. Young Scholars with at least 280 hours in the 
program averaged double-digit increases annually for proficiency in both math and 
reading. Furthermore, the number of Young Scholars receiving A’s and B’s increased an 
average of 38 percent, while the number receiving F’s decreased an average of 50 
percent. 

> Children in the Ohio Urban School Initiative School Age Child Care Project (SACC) 
scored higher in every subject area tested (writing, math, reading, citizenship and 
science) than non-participating students from across the state, according to an evaluation 
by the University of Cincinnati College of Education’s Evaluation Services Center. 

> Policy Studies Associates’ second- year evaluation of The After-School Corporation’s 
(TASC’s) program found significant differences in proficiency-level shifts among active 
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participants and nonparticipants who scored in the lowest proficiency level on the 1998- 
99 mathematics tests. When tested in 1999-2000, participants were more likely than non- 
participants to score at a higher proficiency level as compared to their 1998-99 
performance. A similar but less pronounced pattern was observed on the reading tests 
administered in grades 3-8. 

> Reading scores for San Diego’s “6 to 6” students improved, according to evaluator 
WestEd. Fifty-seven percent of students increased their reading scores over the course of 
the studied year, and SAT-9 reading scores increased. Nearly ten percent of children 
moved up into the 25 th percentile or higher in 2000 by comparison to 1999 reading 
scores. In addition, 44 percent of students in San Diego’s “6 to 6” program increased 
their SAT-9 math scores. 

> A five-site evaluation of the Boys & Girls Clubs’ national Project Learn program found 
average grade increases over the 30-month study period were greatest for program youth, 
who increased their average grades by 1 1 percent while comparison youth increased their 
average grades by only 0.4 percent. 

> Students participating in Foundations, Inc. afterschool programs scored higher on math, 
reading, and language arts standardized tests, at every grade level, than the comparison 
group. 

> Researchers at Fordham University found that participants in the YMCA of Greater New 
York Virtual Y Program demonstrated statistically significant higher gains in math test 
scores compared to matched nonparticipants. 

Frequency and Duration of Afterschool Participation Increases Benefits 

> According to the Promising After-School Programs Study, a 2007 study of the effect of 
afterschool programs on nearly 3,000 low-income students at 35 high-quality afterschool 
programs across the nation, elementary school students who regularly attended the high- 
quality afterschool programs (alone or in combination with other activities) across two 
years demonstrated significant gains in standardized math test scores, compared to their 
peers who were routinely unsupervised during afterschool hours. 

> Chapin Hall’s study of Chicago’s Afterschool Matters program found that, over their full 
high school careers, students enrolled in the program for three or more semesters and 
those who participated at the highest levels had higher graduation rates and lower dropout 
rates than similar students not in the program. 

> A study conducted for the state of New Hampshire of its state- and federally-funded, 
academically focused afterschool programs found that “Regular attendance seems to 
contribute to student success. ... At the middle school levels, where data were richer, 
more students who attended regularly showed academic improvement than students who 
attended less frequently. However, even students who attended sporadically showed 
improvement.” 

> LA’s BEST students who participated most frequently and for the longest period of time 
were least likely to drop out of school. 



www.afterschoolalliance.org 



5 



> A five phase evaluation of the Citizen Schools program begun in 2001 found that 
participants with high levels of program exposure were significantly more likely to be on 
track to successful graduation than matched nonparticipants. 

> Policy Studies Associates’ found that students who participated in TASC afterschool 
activities the most consistently and for the longest period of time experienced the greatest 
math gains. 

> A statewide evaluation of California’s After School Education and Safety Program 
(formerly the After School Learning and Safe Neighborhoods Partnerships Program, 
ASLSNPP) by the University of California at Irvine demonstrated mathematics gains 
closely related to individual students’ levels of participation in the program. 

> The Texas State Education Agency found that academic progress was closely linked to 
the extent of participation in afterschool — fewer than half (46 percent) of youth who 
attended 25 percent or less of available afterschool reading tutorials showing improved 
reading ability at the end of the semester, compared to 64 percent of youth who attended 
more than 75 percent of reading tutorials. 

> A Yale University study of afterschool participation found that children who were in the 
highest category of afterschool program attendance had significantly higher reading 
achievement than children in all other care arrangements (parent, sibling/self-care or 
some combination) while children at lower levels of participation outperformed children 
in only some of the other care arrangements. 

Students at Greatest Risk Show Greatest Gains 

> A five phase evaluation of the Citizen Schools program found that former Citizen School 
participants were more likely to pass the tenth-grade Mathematics and English/Language 
Arts MCAS tests than were students district-wide. This finding is particularly noteworthy 
considering that participants as a group were more academically at-risk than the general 
Boston Public School population at baseline. 

> According to a 10 year study of LA’s BEST, participation appears to have a significant effect 
in reducing the hazard of dropping out for low-income students; that is, the greater the low- 
income status at the baseline, the longer participation in LA’s BEST will keep these students 
in school. 

> A Public/Private Ventures evaluation of the “Communities Organizing Resources to Advance 
Learning” project, an eight-year $58 million afterschool initiative of the James Irvine 
Foundation, concluded that the five-city program’s tightly focused literacy programming 
three to four days a week produced “pronounced gains in achievement for a range of 
students.” Most afterschool students were elementary-school aged, more than half were 
designated English language learners and 89 percent were recipients of free or reduced-price 
lunch. The reading gains were greatest for participant youth who were two or more grade 
levels behind at the time of the first assessment. This improvement represents approximately 
three quarters of a grade level in reading (0.78). 

> A 2006 meta-analysis synthesizing 35 out-of-school time (OST) afterschool program 
studies, conducted by Mid-continent Research for Education and Learning (McREL) with 
funding from the U.S. Department of Education, found that afterschool programs had 
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positive and significant effects among students at risk of failure in reading or math, 
producing positive results on reading achievement, particularly in lower elementary grade 
levels and in high school students. Researchers also found positive and significant effects 
on math achievement, particularly for middle and high school students. 

> The TASC program evaluation concluded that participants who were at greatest academic 
risk made the largest math gains, when compared to other students. Math benefits were 
most clearly evident for students who scored in the lowest of four proficiency levels in 
the year prior to TASC participation. Furthermore, among students from low-income 
families, the evaluation also found evidence of afterschool benefits in math after two or 
more years of active participation. 

> North Carolina’s Support Our Students’ participants who were the furthest behind and 
had the most risk factors (e.g., free/reduced lunch status, single-parent households, etc.) 
made the greatest gains on their End of Grade (EOG) Achievement Test. 

> An external evaluation of Adams County, Pennsylvania’s Generacion Diez afterschool 
program found significantly greater gains in a number of areas (spelling achievement, 
math achievement, reading achievement) for participating children whose families were 
less engaged in school and/or were functioning at a lower level. 



Afterschool Evaluations in Detail 



Over the past decade a number of important afterschool program evaluations have been 
conducted — more than enough to demonstrate that quality afterschool programs help children 
achieve. Following are summaries of some of the most recent evaluations. The summaries are 
organized by scope, beginning with a meta-analysis covering evaluations of afterschool 
programs and evaluations of programs that are national in scope. We then turn to evaluations of 
programs operating at a state level and end with local or program level evaluations. Studies that 
are newly included or updated for this 2008 version of the evaluations backgrounder are 
indicated as such. 



Large-Scale Studies 




The Promising After-School Programs Study 



A study by researchers at the University of California, Irvine, the University of Wisconsin- 
Madison and Policy Studies Associates, Inc. examined the academic impact of high-quality 
afterschool programs on a group of nearly 3,000 low-income students at 35 programs in Aurora, 
Colorado; Baldwin, Michigan; Bridgeport, Connecticut; Central Falls, Rhode Island; Denver, 
Colorado; Los Angeles, California; Missoula, Montana; New York, New York; Oakland, 
California; Pawtucket, Rhode Island; Salem, Oregon; San Diego, California; San Ysidro, 
California; and Seaside, California. The study, conducted with support from the Charles Stewart 
Mott Foundation, was released in October 2007. 
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Noting that many afterschool participants also took part in other structured or adult-supervised 
afternoon activities one or more days a week, researchers compared results across three groups 
of students: children in afterschool programs (“program only”; children in afterschool programs 
two to three days a week and in other structured or adult-supervised activity the other days of the 
week (“program plus”); and children who were frequently unsupervised in the afternoons 
(“unsupervised”). 

In screening programs for inclusion in the study, researchers focused on “high quality” 
programs. Researchers described the screening for quality thusly: 

“Using a rating system, researchers assessed programs based on evidence of supportive 
relationships between staff and child participants and among participants, and on evidence of 
rich and varied academic support, recreation, arts opportunities and other enrichment activities. 
Ratings were consistently positive. Students typically were highly engaged with one another and 
with program activities, and group leaders structured activities to maximize learning and positive 
relationships. Adults facilitated activities without imposing controls that limited student learning 
opportunities. Disruptive or chaotic behavior was rarely observed; when behavioral disruptions 
occurred, leaders managed them calmly and constructively.” [page 2] [This and subsequent 
quotes from “Outcomes Linked to High-Quality Afterschool Programs: Longitudinal Findings 
from the Study of Promising Afterschool Programs,” Deborah Lowe Vandell, University of 
California, Irvine, Elizabeth R. Reisner, Policy Studies Associates, Inc. and Kim M. Pierce, 
University of California, Irvine available at 

http://www.policvstudies.com/studics/vouth/Promising%20Programs%20FINAL.pdl' . | 
Researchers gathered two years of data, and found that: 

■ “regular participation in high-quality afterschool programs is linked to significant gains in 
standardized test scores and work habits as well as reductions in behavior problems among 
disadvantaged students.” [page 1] 

■ “Elementary school students who regularly attended the high-quality afterschool programs 
(alone or in combination with other activities) across two years demonstrated significant gains 
in standardized math test scores, compared to their peers who were routinely unsupervised 
during afterschool hours.” [page 5] 

■ “Program Only and Program Plus students also posted gains in teacher reports of work habits 
(effect sizes of .31 and .35, respectively) and task persistence (.23 and .30, respectively) over 
the two-year period.” [page 5] 

■ “Program Only and Program Plus students posted significant gains in teachers’ reports of 
students’ social skills with peers. . .and prosocial behaviors. . .. Program Only and Program 
Plus students also posted significant reductions in aggressive behaviors with peers. . .” [page 
5] 

■ “Middle school students who regularly attended the high-quality afterschool programs (alone 
or in combination with other activities) across two years demonstrated significant gains in 
standardized math test scores, compared to their peers who were routinely unsupervised 
during afterschool hours. Regular participation in the programs was associated with gains of 
12 percentiles in math achievement test scores over the two-year period, relative to students 
who were routinely unsupervised after school.” [page 6] 



www.afterschoolaUiance.org 



8 



■ “Reductions in misconduct over the two-year period were reported by Program Plus and 
Program Only middle school students, relative to the Low Supervision group.” [page 6] 

■ “Middle school students who regularly participated in afterschool programs also reported 
reduced use of drugs and alcohol, compared to those in the Low Supervision group.” [page 6] 



CASEL’s Meta-Analysis of the Impact of After- School Programs that Promote 
Personal and Social Skills 

The Impact of After-School Programs that Promote Personal and Social Skills study, by Roger 
P. Weissberg, President of the University of Illinois at Chicago-based Collaborative for 
Academic, Social, and Emotional Learning (CASEL) and Professor Joseph Durlak, Loyola 
University Chicago, was intended to fill what the authors regarded as a gap in existing research. 
While many studies have focused on afterschool’s impact on students’ academic achievement, 
large-scale studies focused on programs’ impact on students’ personal and social skills were less 
common. Their meta-analysis, released in 2007 with funding from the William T. Grant 
Foundation, used data from 73 existing studies to identify broad conclusions. The two they 
deemed most significant: 

■ “Youth who participate in after-school programs improve significantly in three major areas: 
feelings and attitudes, indicators of behavioral adjustment, and school performance. More 
specifically, after-school programs succeeded in improving youths’ feelings of self- 
confidence and self-esteem, school bonding (positive feelings and attitudes toward school), 
positive social behaviors, school grades and achievement test scores. They also reduced 
problem behaviors (e.g., aggression, noncompliance and conduct problems) and drug use. In 
sum, after-school programs produced multiple benefits that pertain to youths’ personal, social 
and academic life. 

■ “It was possible to identify effective programs: Programs that used evidence-based skill 
training approaches were consistently successful in producing multiple benefits for youth, 
while those that did not use such procedures were not successful in any outcome area.” [All 
quotations from “The Impact of Afterschool Programs that Promote Personal and Social 
Skills,” Joseph A. Durlak, Loyola University Chicago; Roger P. Weissberg, University of 
Illinois Chicago; Collaborative for Academic, Social, and Emotional Learning (CASEL), 
2007, available at http://www.casel.org/downloads/ASP-Full.pdf.] 

The authors considered programs to be using “evidence-based skill training approaches” if they 
used a “sequenced set of activities to achieve skill objectives,” “active forms of learning,” and at 
least one program element “focused on developing personal or social skills” that targeted 
specific skills. 
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The 2006 McREL Meta-Analysis of Effects of Out-of-School Time Programs for At- 
Risk Students 

In the summer of 2006, Mid-continent Research for Education and Learning (McREL) released a 
meta-analysis of 35 separate studies of out-of-school time programs, after sifting through 
hundreds of studies spanning many years. The study was conducted with funding from the U.S. 
Department of Education. A meta-analysis is a statistical re-analysis of the results of several 
studies on a related topic, conducted for the purpose of integrating the studies’ findings, 
notwithstanding differences in their respective research methodologies. The studies selected for 
inclusion in the McREL meta-analysis all met rigorous methodological standards. 

The studies included in the meta-analysis all employed control or comparison groups to estimate 
the effects of afterschool programs on students at risk of failure in reading or math. The study 
was published in The Review of Educational Research, 76, 275-313, available for a fee at 
http://rer.sagepub.com/content/vol76/issue2/ , and is summarized on the Harvard Family 
Research Project’s website at http://www.hfrp.org/out-of-school-time/ost-database- 
bibliography/database/meta-analysis-of-effects-of-out-of-school-time-programs-for-at-risk- 
students . 1 Researchers included Patricia A. Lauer, Rocky Mountain Center for Health 
Promotion and Education; Motoko Akiba, University of Missouri, Columbia; Stephanie B. 
Wilkerson, Magnolia Consulting, LLC; Helen S. Apthorp and Mya L. Martin-Glenn, Mid- 
continent Research for Education and Learning (McREL); David Snow, Billings Public Schools. 
This same team conducted a 2004 McREL meta-analysis summarized later in this document. 

Researchers found that programs had positive and significant effects on reading achievement, 
particularly in lower elementary grade levels, and in high school students. They also found 
positive and significant effects on math achievement, particularly for middle and high school 
students. 

Researchers conclude: 

> “CfST programs can have positive effects on the achievement of at-risk students in reading 
and mathematics.... [italics in original] Our results contrast with the first-year evaluation of 
the 21 st Century Community Learning Centers (U.S. Department of Education, 2003), which 
found no statistically significant effects of after-school programs on reading or mathematics 
achievement for elementary or middle school students. However, the 21 st Century evaluation 
documented great variation in the characteristics of centers across school districts, 
particularly in the range of activities offered and in the emphasis on academic assistance. 
Conclusions about the ineffectiveness of that program might be due to the aggregation of 
interventions that have different characteristics in the evaluation study. Our synthesis results 
indicate that program duration and student grouping influence program effectiveness. 



1 The Harvard Family Research Project (HFRP) maintains a rich database of information on out-of-school-time 
evaluations, from which much information in this report is drawn. A complete listing of HFRP’s summaries is 
available at http://www.gse.harvard.edu/hfrp/projects/afterschool/evaldatabase.html. 
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Aggregating results across programs that vary in these characteristics can mask positive 
outcomes.” [All bullets quoted from The Review of Educational Research, 76, p. 307] 

> “Students in both elementary and secondary grades can benefit from OST programs for 
improved reading; in contrast, there are indications that benefits for mathematics 
achievement occur primarily in the secondary grades .” 

> “OST programs need not focus solely on academic activities to have positive effects on 
student achievement. Study results indicate that OST programs in which activities are both 
academic and social can have positive influences on student achievement. This finding 
supports the belief that OST programs should address the developmental needs of the whole 
child...” 

Evaluations of State-Level Initiatives and Programs 




New Hampshire Study of Academically Focused Afterschool Programs 



In an effort to facilitate collection and analysis of data about afterschool programs’ impact in the 
state, the Governor of New Hampshire created a task force to develop a common reporting 
program for state- and federally-funded programs. In 2005, the task force published its first 
report based on the resulting data, “Afterschool Learning: A Study of Academically Focused 
Afterschool Programs in New Hampshire,” by Susan L. Frankel, Ph.D., Kim Streitburger and 
Elizabeth Goldman of Portsmouth-based RMC Research Corporation. 



The study examined student outcomes at programs that focused on academics and that were 
funded by the 21 st Century Community Learning Centers initiative and/or the state’s Out-of- 
School Matters! New Hampshire. Research methods included gathering attendance records from 
programs, surveying teachers and students about behavior and academic achievement, and 
surveying afterschool program directors about their staffing and program characteristics. 



The studied programs had varying curricula, but each shared three common components, authors 
said: “homework and tutoring; hands-on projects, service learning, and other enrichment 
activities to develop skills that support and deepen in-school learning; and non-academic 
activities, from sports to dance to drama club, that build positive relationships and improve 
critical thinking skills, motivation, and self-confidence.” Findings were based on data from 
afterschool programs in 16 elementary and 13 middle schools, serving nearly 3,000 students in 
all. Among key findings: 



“More than half the students who attended regularly improved both academically and 
behaviorally.” [All quotes from “Afterschool Learning: A Study of Academically Locused 
Afterschool Programs in New Hampshire,” by Susan L. Lrankel, Ph.D., Kim Streitburger and 
Elizabeth Goldman, available online at http://www.plustime.org/ pdf/Plus Time Programs.pdf . 1 



■ “Afterschool programs improve students’ learning skills. Classroom teachers reported that 
students made the most progress in turning in homework on time, completing homework to 
teachers’ satisfaction, and participating in class.” 
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■ “Regular attendance seems to contribute to student success. . .. At the middle school level, 
where data were richer, more students who attended regularly showed academic improvement 
than students who attended less frequently. However, even students who attended sporadically 
showed improvement.” 

Young Scholars Program - North Carolina 

In 1999, the Z. Smith Reynolds Foundation (ZSRF) came to the conclusion that extended day 
programs could go far beyond their original goals and not only provide supervision during 
afterschool hours but also provide an academic boost. Working with the Public School Forum, 
ZSRF envisioned a program that would combine the best elements of traditional extended day 
programs with academic support that would engage young people’s minds. The result was the 
Young Scholars Program. From 2000-2005, Young Scholars Programs across the state of North 
Carolina served nearly 1,000 young people in 19 elementary and middle schools. 

In August 2006, ZSRF released a report documenting the program’s outcomes. That report, 
available at http://www.nccap.net/Summer2006 ForumReport.pdf , found that: 

> “Young Scholars participants made significant gains on the state’s ABC tests. Young 
Scholars with at least 280 hours in the program averaged double-digit increases annually for 
proficiency in both math and reading. Throughout the five-year period, participants in 
Young Scholar programs consistently exceeded the state’s expected rate of academic 
growth.” [All bullets quoted from Young Scholars Program: An Over\>iew of the Benefits that 
Promising Students Gain From Extended Day Programs ] 

> “Prior to participating in Young Scholars, many of the participants were retained or required 
to repeat a grade each year. Over the five year time covered by this study, promotion rates 
for Young Scholars improved by 83 percent.” 

> “Young Scholars school attendance improved substantially — absences among participants 
decreased by 48 percent.” 

> “Young Scholars participation during the regular day increased dramatically, as did their 
grades. The number of Young Scholars receiving A’s and B’s increased an average of 38 
percent while the number receiving F’s decreased an average of 50 percent.” 

> “Young Scholars parents became far more involved in school activities. The longer students 
were involved in the program, the more involved parents became.” 



The Massachusetts After-School Research Study (MARS) 

In 2005, the Intercultural Center for Research in Education and the National Institute on Out-of- 
School Time at Wellesley College published the results of the Massachusetts After-School 
Research Study (MARS), which examined the track records of 78 different Massachusetts 
afterschool centers serving more than 4,100 children. The study’s objective was to “identify 
those program characteristics that are most closely related to high quality implementation, and to 
explore the links between program quality and youth outcomes.” [All quotes from “MARS,” 
available at http://www.wcwonline.Org/mars/MARSfulLpdf .l 
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Among the study’s findings: 

> “Staff make a difference in program quality, and staff who have a strong educational 
background and appropriate training are key to program quality. At the same time, given 
current wages and working conditions, many programs can not attract such staff, or even if 
recruited, lose a significant portion of staff each year.” 

> “A highly qualified coordinator or program director is key to setting the tone for a program 
that promotes youth engagement, staff engagement, and the quality of activities and 
homework.” 

> “Most programs in the MARS sample had very low staff-to-child ratios, typically between 
1:7 and 1:9. We found clear links between low ratios and high quality, as has previous 
research in the field. At the same time, where ratios are very low, programs may want to 
consider the trade-off between paying higher salaries and having additional staff.” 

> “One of the key youth outcomes — relations with adults — was positively associated with the 
quality of family relations we observed at pick-up time. Programs rated high on this 
component of the APT also tended to cite parent, community and volunteer support as 
strengths of their programs. This finding suggests that all afterschool providers could benefit 
from paying attention to their relationships with the families of the youth in their programs.” 

21 st Century Community Learning Centers -Texas 

At the end of the 2003-2004 school year, Texas had 32 21 st Century Community Learning Center 
grantees, operating 136 afterschool sites. Relying on pre-program and post-program data 
collected from sites as part of grant reporting requirements, the Texas Education Agency, the 
state’s department of education, compiled and analyzed data on student performance and 
program implementation. A summary of the evaluation’s findings by the Harvard Family 
Research Project is available at http://www.hfrp.org/out-of-school-time/ost-database- 
bibliographv/database/21st-centurv-communitv-learning-centers-texas/evaluation-2003-2004- 
evaluation-of-2003 -04-proj ects . The complete evaluation study can be accessed online at 
http://www.tea.state.tx.us/opge/progeval/OutOfSchoolLearning/21cclc 03 04 evakpdf . Among 
the TEA’s chief findings: 

Among participating students, academic progress was closely linked to the extent of participation 
in afterschool. “Less than half (46 percent) of youth who attended 25 percent or less of available 
tutorials showed improved reading ability at the end of the semester, compared to 64 percent of 
youth who attended more than 75 percent of reading tutorials. Conversely, 40 percent of 
participants who attended less than one quarter of the fall reading tutorials experienced a decline 
in their reading ability, as measured by pretests and posttests, compared to just 20 percent of 
youth who attended more than 75 percent of the reading tutorials. Similar results were observed 
for reading in the spring 2004 term. After controlling for demographic factors, youth who 
participated in 26 percent to 50 percent, 51 percent to 75 percent, and over 75 percent of 
available spring reading tutorials showed more increased reading ability than youth attending 
less than 25 percent. These relationships were all statistically significant (p < .05).” [“A Profile 
of the Evaluation of 21st Century Community Learning Centers — Texas,” page 9, Harvard 
Family Research Project, http://www.hfrp.org/out-of-school-time/ost-database- 
bibliographv/database/21st-centurv-communitv-learning-centers-texas/evaluation-2003-2004- 



www.afterschoolalliance.org 



13 



evaluation-of-2003 -04-proi ects . 1 Similar findings were reported for mathematics and science, 
especially during the fall semester. 

Strong participation in afterschool correlated with better attendance during the regular school 
day. “Approximately half (48 percent) of youth who participated in three quarters or more of the 
available 21 st CCLC activities missed five or fewer days of school during the fall semester, 
compared to 17 percent of youth who participated in less than one quarter of the available 
activities. The corresponding percentages for spring were 33 percent and 26 percent, 
respectively. Youth who participated in 50 percent or more of the available 21 st CCLC activities 
were absent approximately two regular school days less in the spring term than youth who 
participated in less than 50 percent of the available 21 st CCLC activities. This difference was 
statistically significant (p < .01). After controlling for demographic factors, those youth 
participating in 26 percent-50 percent, 51 percent-75 percent, and 75 percent-100 percent of 
available activities missed significantly fewer days of school than those participating in 25 
percent or fewer activities (p < .01).” [HFRP Profile, page 10.] 

After School Education and Safety Program - California 

Begun in 1998 as the California Afterschool Learning and Safe Neighborhoods Partnerships 
Program (ASLSNPP), the After School Education and Safety Program (ASES) provides $550 
million annually in matching funds to local partnerships of school districts, community groups, 
and local governments to provide before and afterschool programs for students. In selecting 
grantees, the state gives priority to schools where 50 percent or more of pupils are eligible for 
free or reduced-cost meals. Programs are designed locally, but are required to include an 
educational and literacy component to provide tutoring or homework assistance in one or more 
of the following subject areas: language arts, mathematics, history and social science, or 
science; and an educational enrichment component, which may include but is not limited to, 
recreation and prevention activities. “Such activities might involve the visual and performing 
arts, music, physical activity, health/nutrition promotion, and general recreation; career 
awareness and work preparation activities; community service-learning; and other youth 
development activities based on student needs and interests.” [California’ s After School 
Education and Safety Program Description, at 

http://www.cde.ca.gOv/ls/ba/as/pgmdescription.asr .l In all, 947 afterschool programs were 
funded during the ASES program’s first two years. 

Working with the California Department of Education, the Education Department of the 
University of California at Irvine conducted evaluations of two academic years of the program, 
from 1999 to 2001, releasing results in February 2002. The evaluation relied on data supplied to 
the state by participating programs, as required by law, and examined student and parent 
satisfaction with their programs, as well as students’ academic outcomes. 

Findings included: 

> SAT-9 scores of participating students increased faster than those of students statewide. In 
reading, 4.2 percent of afterschool students moved from out of the lowest 25 percent of their 
classes. “This increase is more than twice the increase found among all students statewide 
(1.9 percent). . .” [Evaluation of California ’s After School Learning and Safe Neighborhoods 
Partnerships Program, Department of Education, University of California at Irvine, February 
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1, 2002, page 4.] In math, similar findings: 2.5 percent of afterschool participants moved 
out of the lowest quartile, compared with 1.9 percent statewide. [ Evaluation , page 6.] 

> Significantly, gains were closely related to individual students’ levels of participation in the 
program. “Among students who participated for more than 150 days (approximately 7.5 
months), there was an increase of 4.9 percent in students above the 25 th percentile, an 
increase considerably larger than that found statewide. The scores suggest that, particularly 
for students who participate in the ASLSNPP for substantial periods of time, there is a 
closing of the gap in Math achievement between low -income and other students .” 

[Evaluation, page 6. Emphasis in original.] 

> “The regular school day attendance of students in the ASLSNPP increased between 1999 and 
2000. Among the ASLSNPP participants who were absent 5 percent or more days in 1999, 
the average increase in attendance was 5.6 days. Among those who were absent 10 percent 
or more days in 1999, the average increase in attendance was 11 days. Lor those absent 15 
percent or more days in 1999, the average increase in attendance was 17 days.” [Harvard 
Lamily Research Project, Summary of ASLSNPP Evaluation, http://www.hfrp.org/out-of- 
school-time/ost-database-bibliography/database/after-school-education-and-safety-program- 
california , September 3, 2002.] 



The After-School Corporation (TASC) - New York 

TASC supports 322 programs across New York City serving more than 300,000 kids, and helped 
establish programs for another 50,000 kids in the New York region. TASC’s mission is to 
enhance the quality, availability and sustainability of afterschool programming in New York City 
and beyond, and change public policy so that every child in every community across the nation 
will have access to free, quality after-school programming by 2010. TASC began in 1998 with a 
multi-year challenge grant from the Open Society Institute (OSI) of up to $125 million, based on 
the condition that for every dollar OSI contributes, TASC is responsible for raising three dollars 
in other public and private monies. To date, TASC has exceeded their goal and leveraged $538 
million in public and private funds to support TASC programs. TASC provides grants to 
nonprofit organizations to establish partnerships with individual public schools, and the resulting 
afterschool programs follow a core set of program components. 

The Charles Stewart Mott Loundation and the Carnegie Corporation, joined by the W.T. Grant 
Loundation and the Atlantic Philanthropies provided funding to the Washington-based Policy 
Studies Associates to conduct a five-year evaluation, including annual summary reports. 

The final installment of the Policy Studies evaluation of TASC covered data from four full years 
of program operations and was completed in 2004. When all was said and done, researchers 
concluded: 

> “School principals reported significant benefits for students who participated in TASC 

programs. Ninety-five percent said that TASC gave students access to activities not available 
during the regular school day, 79 percent said that participants’ parents expressed more 
positive feelings about the school than before, and 66 percent said that after-school 
participants received special opportunities to hone literacy skills.” [This and subsequent 
quotations from this study are from Building Quality, Scale, and Effectiveness in After- 
School Programs: Major Findings of the TASC Evaluation, Policy Studies Associates, Inc., 
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page 3, available at 

http://www.policystudies.com/studies/youth/TASC%20Summary%20Report%20Final.pdf.] 

> “Parents also expressed satisfaction with TASC services, based, as reported in surveys 
administered to a convenience sample of parents, on their children’s enjoyment of program 
activities, their children’s completion of homework before coming home, and the reassurance 
of knowing that their children were safe and cared for after school, which allowed parents to 
work more hours and miss work less often.” 

> Analyses of data on academic performance and school attendance show that participation in 
TASC activities was linked to improvements in both areas, especially for students who 
participated regularly in TASC programming over two consecutive years. At the elementary- 
and middle-grades level: 

■ “Analyses indicate after-school benefits in mathematics achievement, especially for 
those participants who attended TASC programs regularly and for more than a 
year. ... Comparable findings in reading and English language arts were not apparent, 
although participants in some programs consistently out-gained comparable 
nonparticipants in this area.” 

■ “Participants showed greater gains in school attendance than did nonparticipants, with 
the greatest benefits at grades 5-8.” 

At the high school level: 

■ “After-school participants passed more Regents exams and earned more high school 
credits than nonparticipants, but conclusions from this evidence are limited by the 
fact that high school participants were already achieving at substantially higher levels 
than nonparticipants before entering a TASC program.” 

■ “The difference that reflects the clearest association with TASC high school 
participation is in improved school attendance. When compared to nonparticipants 
with similar records of prior school attendance, TASC participants showed 
significantly more positive school attendance after a year of TASC participation 
(although the attendance of both participants and nonparticipants declined over grades 
9-12). For example, the difference in the attendance gain of participants and similar 
nonparticipants in the lowest attendance quartile during the year before enrolling in 
TASC was +2.4 percentage points over one year, or the equivalent of a net gain of 
+4.4 school days in a 181-day school year (effect size of +0.27).” 

North Carolina’s ‘Support Our Students’ 

In 1994, the state of North Carolina launched its “Support Our Students” initiative (SOS), to 
provide funding for afterschool programs across the state. The program offers grants in the 
$60,000 to $250,000 range to nonprofit organizations in the state — one per county, each of 
which coordinates services in their counties. In 2001-2002, the program provided $12.5 million 
to nonprofits in 98 counties. In that same year, the program supported programs in 190 school- 
based sites, and 54 community-based sites, providing afterschool services to 16,000 students 
during the school year, and summer programming for 10,000 students. By 2003-2004, the 
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program served students in all 100 counties and raised around $10 million from other grants, in- 
kind donations, and volunteer labor valued at more than $1 million. 

An evaluation of the 2003-2004 year’s programs, conducted by EDSTAR, an independent 
research and analysis firm based in Raleigh, North Carolina, found the following: 

> “Among students who participated in SOS Programs, 89 percent scored at grade level on 
their End-of-Grade Math tests, and 81 percent scored at grade level on their End-of-Grade 
Reading tests,” exceeding statewide increases in scores. [All quotes for this study are from 
North Carolina’s Support Our Students 2003-2004 Program Highlights , at 
http://www.edstar.biz/edstar/reports/file/SOS_2004_highlight.doc] 

> Classroom teachers reported that more than 40 percent of the regularly attending participants 
improved their grade in English and/or math. 

> “35 percent of middle school students and 49 percent of elementary students reported that 
they enjoy school more since joining the SOS Program.” 

> “Nearly a third of the SOS Program participants’ math and English teachers reported 
improved classroom behavior.” 



Findings from an evaluation of the 2001-2002 school year also show positive results: 

> “The percentages of students who scored at grade level proficiency increased in both reading 
and math, with the greatest increase in reading — from 67 percent at grade level to 7 1 
percent.” [A summary of the evaluation’s findings by the Harvard Family Research Project is 
available at http://www.hfrp.org/out-of-school-time/ost-database- 

bibliographv/database/north-carolina-support-our-students-sos-initiative/evaluation-2001- 
2002 . This and subsequent quotes for this study are from the HFRP summary.] 

> “Minority students made greater improvements than White students in both math and 
reading.” 

> “African Americans made greater improvements in reading than any other demographic 
group; minority participants overall made nearly twice the gain of White students in reading 
scores.” 

> “Students continued to make steady, consistent improvement each year they participated in 
SOS. The average yearly improvement was slightly less than half a proficiency level.” 

> “Of the three-year SOS participants, more than two thirds had improved at least two 
proficiency levels in reading and math, compared with the year before joining SOS.” 

> “Except for sixth graders, SOS participants who were the furthest behind and had the most 
risk factors (e.g., free/reduced lunch status, single-parent households, etc.) made the greatest 
gains on EOG. Evaluators surmised that sixth graders often had problems making the 
transition to middle school, and recommended that sixth-grade transition programs be 
implemented.” 
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Ohio Urban School Initiative School Age Child Care Project 

The Urban School Initiative School Age Child Care Project (SACC) funds a variety of 
afterschool programs in Ohio urban school districts. The University of Cincinnati College of 
Education’s Evaluation Services Center conducted a thorough review of the program’s 1998- 
1999 school year, measuring both project design and its outcomes. Data collection included 
document reviews, observation of programs, surveys and questionnaires. Among the findings: 

> “Ohio Proficiency Tests scores for both 4 th and 6 th graders showed that SACC children 
exceeded the state- wide percentages of students meeting proficiency standards. SACC 4 th 
grade students’ scores exceeded the statewide percentages of students meeting proficiency 
standards in every subject area tested: writing, reading mathematics, citizenship, and science. 
SACC 6 th graders exceeded the statewide percentages of students meeting proficiency 
standards in four of the five areas: writing, reading mathematics, and citizenship.” [This and 
subsequent quotes for this study are from Harvard Family Research Project at 
http://www.hfrp.org/out-of-school-time/ost-database-bibliography/database/ohio-urban- 
school-initiative-school-age-child-care-proiect .l 

> “School absence and tardiness were reduced for participating students. First graders who 
were not in a SACC program during kindergarten reduced the number of school days they 
missed from an average of eight during their kindergarten year to an average of three days 
during their 1998-99 1st grade year. Eighth graders who were not in a SACC program during 
7th grade reduced the average number of school days missed from 18 to five.” 

> “Parents participating in interviews or completing surveys felt the programs had positive 
impacts on their families.” 



Local or Program Level Evaluations 




Public/Private Ventures Evaluation of CORAL 



In 1999, the James Irvine Foundation launched an eight-year, $58 million afterschool initiative to 
improve the educational performance of low-achieving students in five California cities. 
“Communities Organizing Resources to Advance Learning,” (CORAL), funded programs in the 
lowest-performing schools in Fresno, Long Beach, Pasadena, Sacramento and San Jose. 



In 2004, the Foundation sharpened the focus of the initiative and brought Public/Private Ventures 
(P/PV) into the project to help with intensifying the academic focus, and to evaluate program 
effectiveness. P/PV’s evaluation commenced with the 2004-2005 school year, during which time 
total CORAL enrollment statewide was 5,321 students across the five cities. Most were 
elementary-school aged, more than half were designated English learners and 89 percent were 
recipients of free or reduced-price lunch. 



At the start of the evaluation, the students’ scores on the California Standards Test-English 
Language Arts (CST-ELA), from spring 2004, showed that only a small portion (16 percent) met 
or exceeded the grade-level proficiency standards for reading. Of the sample explored in most 
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depth in this evaluation, 50 percent were reading two or more grades below level, and an 
additional 20 percent were reading one grade below level. 

In fall 2004, CORAL programs began offering a tightly focused literacy program three to four 
days a week, for 60 to 90 minutes each day. Programs relied on what P/PV describes as 
“concrete strategies for helping children far behind in reading skills improve, while also bringing 
literature and reading to life for the children. This programming was based on the core strategies 
of balanced literacy, a model that reflects current research on developing competent readers.” It 
included such activities as “read-alouds,” book discussions, writing, vocabulary, skill 
development and independent reading, [page iv.] [This and subsequent quotes for this study are 
from Advancing Achievement: Findings from an Independent Evaluation of a Major After- 
School Initiative available online at 

http://www.ppv. org/ppv/youth/youth_publications.asp?section_id=8#pub225.] 

“These changes led to pronounced gains in achievement for a range of students,” according to 
evaluators, [page iii.] Specifically, they found: 

> “Children’s reading success was strongly related to the quality of literacy programming. 
CORAL participants showed greater gains in grade-level reading and performed better on 
standardized tests when they were exposed to more consistent and higher-quality literacy 
activities. Results from the first year of the evaluation indicated greater gains over five 
months on the individualized reading assessment (.45 grade-levels in reading) for children 
exposed to consistent implementation of the balanced literacy strategies — read-alouds, book 
discussions, independent reading, writing, vocabulary development and skill development 
activities — and higher-quality implementation of those strategies. In contrast, those children 
exposed to inconsistent or low-quality implementation of the literacy strategies gained just .26 
grade-levels in reading.” [page v.] 

> “In the second year of the evaluation, when almost all of the staff leading the literacy 
activities had improved and were consistently using the literacy strategies, the average reading 
gain for all children in the sample (based on the individualized reading assessment) was .44 
grade levels — comparable to the average gain of .45 for children exposed to higher-quality 
activities during Year One. Also in Year Two, children in groups where team leaders used 
stronger classroom practices (the instructor offered strong adult support, was skilled at group 
management, provided high-quality instruction and made connections between the children’s 
lives and the books they were reading) in combination with more consistent and higher- 
quality implementation of the literacy strategies were more likely to have a positive outcome 
on the CST-ELA [California Standards Test-English Language Arts] test.” [page v.] 

> “The stronger the children’s sense of belonging to the program, the more likely they were to 
have a positive change in outcomes that included enjoyment of reading, liking and wanting to 
go to school, and time spent reading after school.” [page v.] 
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Chapin Hall Study of After School Matters 

Chicago’s “After School Matters” program was created in 2000, as an after school job training 
program, initially focused on the arts, but subsequently expanded to include sports, technology 
and communications. Led by Maggie Daley, wife of Mayor Richard Daley, Jr., the program 
offers paid internships to teenagers in some of Chicago’s poorest neighborhoods. As of 2006, the 
program, working with its partners, the City of Chicago, the Chicago Public Schools (CPS), the 
Chicago Park District, and the Chicago Public Library, had grown to include 725 programs in 35 
schools, providing more than 22,000 apprenticeships and other opportunities for teens. 

The program requires students to attend school during the day in order to participate in the paid 
internship program. A 2006 study, “After-School Programs and Academic Impact: A Study of 
Chicago’s After School Matters,” [available at 

http://www.chapinhall.org/content director.aspx?arid=1444&afid=335&dt=11 by researchers 
Robert Goerge, Gretchen R. Cusick, Miriam Wasserman, and Robert Matthew Gladden, all of 
the Chapin Hall Center for Children at the University of Chicago, examined the program’s 
impact on daily attendance and performance of participating students. 

■ After accounting for student demographic characteristics and prior attendance records, 
researchers found that students who participated in the program missed fewer days of school 
than their classmates, and that students who participated most frequently failed fewer core 
academic courses (English, Math, Science, and Social Studies). 

■ Over their full high school careers, students enrolled in the program for three or more 
semesters and those who participated at the highest levels had higher graduation rates and 
lower dropout rates than similar students not in the program. 

Authors noted that the finding of improved academic performance is particularly noteworthy 
because the program is not aimed at improving academics. Rather, it focuses on improving 
students’ work skills. But by creating an incentive for students to attend school regularly and 
giving them something to look forward to after school, researchers conclude, After School 
Matters helped improve academic performance, even though the program is not designed 
specifically for that purpose. 



Citizen Schools 

Since 1995, the Boston-based Citizen Schools has provided afterschool and summer programs 
designed to provide children with “authentic, hands-on learning experiences, supportive 
relationships with adults, and positive youth development opportunities.” The program includes 
help with homework, team-building activities, “apprenticeships,” and more. The program is 
based in public schools, and seeks to coordinate its work with school-day teachers. In fall 2002, 
Citizen Schools expanded beyond its home market, to create programs in San Jose, California; 
Houston, Texas; and Worcester and Framingham, Massachusetts. 
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In 2001, Citizen Schools began sponsoring a five-phase evaluation conducted by Policy Studies 
Associates, Inc. The most recent phase of the research was released in January 2008 and is 
available at 

http://www.policystudies.com/studies/youth/Citizen%20Schools%20Phase%20V%20report.pdf. 
“Establishing a Foundation for Progress Toward High School Graduation: Findings from Phase 
V of the Citizen Schools Evaluation,” found: 

> “Former 8 th Grade Academy participants enrolled in high-quality high schools at a 
significantly higher rate than matched nonparticipants. Even though many former 
participants attended high-quality high schools with noteworthy academic demands, 
former participants continued to be academically engaged and successful in ninth and 
tenth grade. Former 8 th Grade Academy participants also attended school at a higher rate 
than matched nonparticipants in ninth and tenth grade. Participants were more likely than 
matched nonparticipants to pass their third-period English course in ninth grade and 
third-period mathematics course in tenth grade. In tenth grade, former 8 th Grade 
Academy participants’ academic success was reflected in MCAS performance. 
Participants outperformed matched nonparticipants on the English/Fanguage Arts MCAS 
assessment.” [All bullet points for this study are quoted from Establishing a Foundation 
for Progress Toward High School Graduation: Findings from Phase V of the Citizen 
Schools Evaluation, page 39.] 

> “Findings from Phase V also show that Citizen Schools is making progress in supporting 
grade promotion and high school graduation. Former 8 th Grade Academy participants 
with high levels of program exposure had a significantly higher rate of on-time 
promotion to tenth grade than did matched nonparticipants. Participants’ progress toward 
on-time graduation looks promising. Participants overall and those with high levels of 
program exposure were significantly more likely to be “on-track” to graduation than 
matched nonparticipants.” 

> “In addition to outperforming a group of matched nonparticipants on several of the 
evaluation’s indicators of high school success, former Citizen School participant were 
more likely to pass the tenth-grade Mathematics and English/Fanguage Arts MCAS tests 
than were students district-wide. This finding is particularly noteworthy considering that 
participants as a group were more academically at-risk than the general BPS [Boston 
Public Schools] population at baseline.” 



The YMCA of Greater New York’s Virtual Y Program 

Since spring 1996, the YMCA of Greater New York has run a Virtual Y initiative — a school- 
based afterschool program serving 2 nd through 4 th graders five afternoons a week. In the 1997- 
98 school year, the program expanded from its initial ten pilot sites to operate in as many as 100 
sites, and no fewer than 66 sites, each year since. More than 50,000 children have participated. 
The program is aimed at lower income public schoolchildren, providing a safe environment 
where children receive reinforcement in reading, math, and healthy lifestyles, as well as training 
intended to emphasize such values as respect, responsibility, honesty and caring. 

The program has undergone several evaluations by the National Center for Schools and 
Communities (NCSC) at Fordham University. In 2005, NCSC published an overview of seven 
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years of research, The Virtual Y Afterschool Program: A Ray of Hope for Urban Public 
Elementary School Children , available at 

http://www.ncscatfordham.org/binarvdata/files/rayofsunshine final.pdf) . The report found: 

> “Across years, teachers reported between 85 and 91 percent of program participants as 
having behavior problems, usually mild. The greatest problems were consistently in the areas 
of task motivation, frustration, tolerance, and learning skills.” [All quotes for this study are 
from The Virtual Y Afterschool Program: A Ray of Hope for Urban Public Elementary 
School Children, pages 8-9.] 

> “In all years, the data showed statistically significant and moderate to large improvements on 
all seven subscales and on the overall behavior scale.” 

> “NCSC analyses found effects for children at each level of behavioral difficulty from mild to 
severe. The likelihood of obtaining chance differences in performance as large as those found 
in this study is less than one percent.” 

> Virtual Y students outperformed a comparison group in attendance and mathematics, 
although not in reading. 

Generacion Diez - Adams County, Pennsylvania 

Adams County, Pennsylvania’s Generacion Diez (G-10) afterschool program is aimed 
specifically at the children of migrant workers in the area. Among its goals for children and their 
families is improving the academic achievement of the participating 1 st through 6 th graders. A 
series of studies conducted by Nathaniel R. Riggs and Mark T. Greenberg, of the Prevention 
Research Center at Pennsylvania State University, summarized by the Harvard Family Research 
Project at http://www.hfrp.org/out-of-school-time/ost-database- 

bibliography/database/generacion-diez , identified a number of positive outcomes, interesting also 
because of the studies’ findings about which students benefited most. The most recent study 
examined data collected between 2001 and 2003. 

Quoting from HFRP’s summary: 

> “From pretest to posttest, participants’ scores increased significantly in reading, spelling, 
and math (p < .01).” 

> “Among the entire sample, more acculturated children showed significantly greater gains 
in reading achievement over the course of the G-10 program (p < .01), as did children 
whose parents reported less engagement in children’s school activities (p < .05). No 
significant differences in achievement were found by parent-teacher contact or family 
functioning.” 

> “For children younger than 8 years old, the following groups showed significantly greater 
gains in reading achievement: more acculturated children (p < .05), children in families 
with lower family functioning (p < .05), children whose parents reported fewer parent- 
teacher contacts (p < .01), and children of parents who reported less engagement in 
children’s school activities (p < .01). For children 8 years old and older, none of the 
assessed variables predicted change in children’s reading achievement.” 
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> “For the entire sample, more acculturated children, children whose families were 
functioning at a lower level, and children of parents who reported less school engagement 
demonstrated significantly greater gains in spelling achievement (p < .05 for each).” 

> “For the entire sample, children whose parents reported less engagement with their 
children’s school activities at pretest showed significantly greater gains in math 
achievement across the program year (p < .01). No significant results were found for 
acculturation, family functioning, or parent-teacher contacts.” 

> “For children younger than 8 years old, children of parents who were less engaged in 
school activities showed significantly greater gains in math achievement (p < .05). No 
significant results were found for acculturation, family functioning, or parent-teacher 
contacts.” 

> “For children 8 years and older, children from lower functioning families and children 
whose parents reported less [engagement] with their children’s school activities showed 
significantly greater gains in math achievement (p <.05 and p <.01, respectively). No 
significant results were found for acculturation or parent-teacher contacts.” 

Mahoney and Lord Ecological Analysis of After-School Program Participation 

One important and increasingly appreciated challenge researchers confront in evaluating the 
success of afterschool programs is sorting out the relative effects of the many different afternoon 
arrangements that parents have settled on for their children. Some children spend all five 
afternoons a week in an afterschool program, others spend all five in the care of parents or 
another adult. But many children spend their afternoons in some combination of settings, often 
shuttling from an afterschool program to some other care. 

In July 2005 Joseph Mahoney and Heather Lord of Yale University and Erica Carryl of New 
York University released the first of several stages of a study aimed at sorting through and 
comparing the effects of these different care arrangements, with an eye toward isolating the 
effects of specific arrangements. This first installment of a longitudinal study focused on 
academic achievement of 599 “students enrolled in the first, second, and third grades of three 
public schools in the Northeastern United States.” [An Ecological Analysis of After-School 
Program Participation and the Development of Academic Performance and Motivational 
Attributes for Disadvantaged Children , Joseph L. Mahoney, Heather Lord, Erica Carryl, in Child 
Development, July/August 2005, Volume 76, Number 4, pp. 811-825, p. 813] When completed, 
the study will track afterschool students for four years, comparing students with four different 
types of afterschool arrangements: those in afterschool programs, those cared for by parents, 
those cared for by a combination of siblings or in self care, and those cared for by a combination 
of other adults and in sibling or self care. 

Students in the study were enrolled in a large urban school district, and most lived in poverty. 

The ongoing study is supported by a grant from the National Institute of Child Health and 
Human Development. 

This first installment found that, “The reading achievement of children in ASP [afterschool 
program] care was significantly higher than of those in each of the three alternative care 
arrangements. Expectancy of success was also significantly higher for children in ASP care 
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compared with children in other adult/non-adult care.” Refining their analysis, the researchers 
also looked at how children’s level of engagement in an afterschool program affected academic 
outcomes, and concluded that, “Children in the high-ASP-engagement subgroup had 
significantly higher reading achievement than did those in the three alterative care 
arrangements,” while children with low-engagement achieved at higher levels than only some of 
the other three groups. 

Other installments of the study are reported on in the Afterschool Alliance’s 

Summary of Formal Evaluations of Afterschool Programs’ Impact on Behavior, Safety and 

Family Life, available on the Afterschool Alliance website. 

LA’s BEST 

Los Angeles’s Better Educated Students for Tomorrow, or LA’s BEST, is among the largest and 
best known afterschool programs in the nation. Launched in 1988 as a partnership between the 
Los Angeles Unified School District (LAUSD), the city of Los Angeles and the private sector, 
the program operates at 180 school sites, serving more than 26,000 students. Schools are chosen 
for participation because of the generally low academic achievement among their students, or 
because of the low economic status of the community, or high gang or crime rates in the 
neighborhood. 

Since early in the life of the program, the UCLA Center for the Study of Evaluation has 
conducted evaluation studies. The studies have focused on a variety of topics, using a range of 
measures. The Center released its separate studies in March 1990, March 1991, July 1991, 
December 1993 and spring 1995. In June 2000, the Center released a comprehensive report 
summarizing each of the five previous studies and adding a rich set of findings based on its five- 
year tracking of the academic performance and school attendance of LA’s BEST students who 
were in 2 nd through 5 th grades in the 1993-94 school year. Then in early 2006, the Center 
released a long-term “effect study,” specifically focused on the long-term impact on students of 
their participation in LA’s BEST, taking advantage of the massive data set collected over the 
years. 

In particular, the 2006 study (“Keeping Kids in School: An LA’s BEST Example: A Study 
Examining the Long-Term Impact of LA’s BEST on Students’ Dropout Rates,” 
https://www.lasbest.org/resourcecenter/Keeping Kids in School Exec Sum.pdf ,) examined the 
dropout rates of students who had participated in LA’s BEST for at least one year, finding 
statistically significant differences between LA’s BEST students and a similar sample of non- 
participating students. Moreover, the study found that the dropout rates of LA’s BEST students 
who participated most frequently and for the longest period of time were most affected: 

“[T]he more regularly the students attended the program, the higher the reduction of the 
hazard of dropping out of school. Most interestingly, LA’s BEST participation also 
appeared to have a significant effect in reducing the hazard of dropping out for low- 
income students; that is, the greater the low-income status at the baseline, the longer 
participation in LA’s BEST will keep these students in school. Since LA’s BEST has 
specifically designed the program to serve low-income students, it is encouraging to see 
the program has a significant effect on the targeted population. 
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The 2000 study had similarly positive conclusions. “To study LA’s BEST schools,” the authors 
wrote, “we obtained information about students including ethnicity, gender, language proficiency 
status, eligibility for free/reduced lunch (the proxy for low-income level) and disability status. In 
addition, we collected outcome data including achievement test scores (using either the 
Comprehensive Test of Basic Skills [CTBS] or the more recently adopted Stanford-9 
Achievement Test [SAT-9] in reading, mathematics, and language arts. The rate at which 
students were redesignated as fully proficient in English was also collected. We also obtained 
school absence rates, course-taking patterns and rates of student mobility (moving between 
schools or out of the district).” 

In addition, because of the size of the LA’s BEST program and of the LAUSD school system, 
researchers were able to track an extraordinarily large sample of students and a correspondingly 
large “control” group: more than 4,000 LA’s BEST students and more than 15,000 non- 
participating students. The sheer numbers of students tracked make the data produced highly 
reliable. 

The findings are powerful evidence of the value of afterschool programming. In short, the study 
found that LA’s BEST participants, defined as students who participated regularly and over a 
period of more than one year, when compared to non-participating students, were absent less 
from school, “show positive achievement on standardized tests in mathematics, reading and 
language arts,” and had “higher language redesignation rates to English proficiency.” [Quoting 
from LA’s BEST’S summary of the findings, at 

http://www.lasbest.org/resourcecenter/ucla_past.php.] Specific findings: 

> “[0]ur results show that higher levels of participation in LA’s BEST led to better subsequent 
school attendance, which in turn related to higher academic achievement on standardized 
tests of mathematics, reading and language arts.” [page 7] [This and subsequent quotes for 
this study are from A Decade of Results: The Impact of the LA ’s BEST After School 
Enrichment Program on Subsequent Student Achievement and Performance, a longitudinal 
study report and a synthesis of research begun in 1990 by the UCLA Center for the Study of 
Evaluation with support from the Bandai Foundation and the City of Los Angeles. June 
2000. Denise Huang, Barry Gribbons, Kyung Sung Kim, Charlotte Lee, Eva L. Baker.] 

> “Language redesignation rates favored LA’s BEST students when compared with non-LA’s 
BEST students for the cohort analyzed (fourth grade, 1994-1995). Significant differences in 
favor of LA’s BEST students were found for subsequent redesignation rates in Grades 6 and 
8. No significant differences were found in comparing performance for Grades 5 and 7.” 
[page 8.] 

> “Absence follow-up data for the fifth-grade cohort (1994-1995) showed that students 
who participated in LA’s BEST had significantly fewer absences in Grades 6 and 7, 
although no differences were detected in Grades 8 and 9.” [page 8.] 

> “Although in the initial year LA’s BEST students began with statistically significant 
mathematics achievement scores lower than those of non-participants, in 1997-1998 
those differences no longer existed.” [page 9.] 
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Researchers conclude: “From our perspective, it looks as if LA’s BEST is a program that, when 
followed as a regular part of students’ broad educational experience, results in statistically 
important differences in student outcomes. The fact that we can detect any change on 
standardized achievement measures in itself is notable, for most educational interventions are 
unable to show impact on measures not tightly tied to the curriculum, or on follow-up 
achievement after a particular program is over. On a practical level, LA’s BEST needs to focus 
its attention on increasing the attendance of enrolled students. It may be that high-level attenders 
do so because they and their parents are more highly motivated, and this interest transfers to 
achievement. But it is equally likely that coming to school and to the LA’s BEST program 
regularly is the reason for good performance and persisting impact subsequent to leaving LA’s 
BEST.” [pages 9-10.1 

Pathways to Progress, St. Paul, Minnesota Public Schools 

Operating with a three-year 21 st Century Community Learning Centers grant, in 2000 the St. 
Paul, Minnesota public schools established Pathways to Progress, an eight-site afterschool 
program. Between 2000 and 2003, the sites served more than 3,000 students, one-third of all 
students enrolled at the eight participating schools. 

Researchers Kyla Wahlstrom, Tim Sheldon and Ashley Murphy of the Center for Applied 
Research and Educational Improvement of the University of Minnesota 
(http://education.umn.edu/CAREEdefault.html) conducted an evaluation of the program, 
assessing its success in achieving its principal objectives, among them, increasing student 
academic achievement. Released in March 2004, the evaluation used a matched-pair technique, 
comparing students who had attended the program for 30 or more days each year with matched 
students who had not participated in the program. It found that: 

> “In reading and math, more Pathways participants scored above the national norm on the 
SAT10 and fewer students scored in the lowest quartile.” [This and subsequent quotes for 
this study are from Final Evaluation Report, 21 st Century Community Learning Centers, 
Pathways to Progress, St. Paul, Minnesota, page 3.] 

> “Data gathered on both two-year and three-year regular participants indicate the groups 
experienced continuous progress in their standardized test scores for both reading and 
math. . . . Regular participants frequently began with lower scores in the years prior to 
Pathways involvement and still completed 2003 with higher scores on the standardized tests 
in both subject areas.” 

> “Pathways students experienced dramatically better school attendance, with participants 
attending 18.44 more school days and missing 9.57 fewer school days than their 
nonparticipant counterparts.” 

> “Middle school students in Pathways generally received better marks in English and math 
and more of the grades received by these students were satisfactory ones — a grade of C 
minus or better.” 
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The Foundations Inc. After School Enrichment Program 

For more than a decade, Foundations, Inc. has operated extended-day enrichment programs and 
provided technical assistance to other afterschool sponsors. During the 2001-2002 school year, 
Drs. Stephen P. Klein and Roger Bolus of Gansk & Associates, of Santa Monica, California, 
administered pre- and post-tests in mathematics and reading to first- through fifth-grade students 
in 19 Foundations programs in three states. A summary of the report is available at 
http://www.childtrends.org/what works/city scan/Philadelphia/Foundations%20Inc. expanded.h 
tm. The full report, issued in December 2002, concludes: 

> “Foundations students made substantial improvements in average scores between the fall 
pretest and spring posttest. In fact, their average score gains in mathematics were somewhat 
greater than what would be expected given the results obtained in CTB/McGraw-Hill’s 
national norm sample. The Foundations students’ gains in reading kept pace with those 
made in this national norm sample.” [page 2] [This and subsequent quotes for 2001-2002 
results are from Improvements in Math and Reading Scores of Students who Did and Did Not 
Participate in the Foundations After School Enrichment Program During the 2001-2002 
School Year , Stephen P. Klein, Ph.D. and Roger Bolus, Ph.D., Gansk & Associates, 
December 2, 2002.] 

> “Over the course of the school year, Foundations’ afterschool students’ test averages moved 
them up national percentile rankings - by an average of 10 percentile rankings in 
mathematics reading, and an average of 2 percentile rankings in reading.” [page 10] 

> “Foundations students fared very well by comparison to non-Foundations students at the 
studied schools. The mathematics “effect size” difference averaged .39 (representing 
39/100 ths of a standard deviation unit), in afterschool students’ favor. In reading, a similar 
finding: a .41 effect size advantage for afterschool students.” [page 14] 

In September 2003, Drs. Klein and Bolus released a follow-up report, assessing data from the 
2002-2003 school year, again finding statistically significant improvement. 

> “The major finding from this analysis is that there was a statistically significant improvement 
in scores between the pretest and posttest at every grade level and in every subject. 

Moreover, the amount of gain was greater than what would be expected given the results in 
the national norm group. For example, the average FOUNDATIONS’ student was at the 
40th percentile in mathematics on the pretest and at the 46 th percentile on the posttest. The 
corresponding values for reading were 45th on the pretest and 48th on the posttest. Language 
arts was 43 ld percentile on the pretest and 48 th on the posttest. Results were similar within 
grade levels.” [page 7] [This and subsequent quotes for 2002-2003 results are from 
Improvements in Basic Skills Scores of Students Who Did and Did Not Participate in the 
Foundations After School Enrichment Program during the 2002-2003 School Year, Stephen 
P. Klein, Ph.D. and Roger Bolus, Ph.D., Gansk & Associates, Santa Monica, California, 
available from Foundations, Inc., by emailing info@foundationsinc.org.] 

> “We conducted regression analyses to assess the Foundations program’s overall effect on 
improving the students’ test scores. These analyses predicted a student’s posttest score on a 
test on the basis of that student’s pretest score on that same test, grade level, and “group” 

(i.e., Foundations versus non-Foundations). These analyses found that the students who 
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participated in the Foundations program had about a 5 -point higher posttest score (which is 
equivalent to about one tenth of a standard deviation unit) than did similarly situated non- 
participants (see Table 6). All the differences in Table 6 were statistically significant (at p < 
.10).” [page 11] 

> “The students who participated in the Foundations program during the 2002-2003 school 
year made substantial gains in math, reading, and language arts scores between the fall 
pretest and spring posttest. These gains generally exceeded the progress of students in 
CTB/McGraw-Hill’s national norm sample. First graders in the Foundations program did 
particularly well, especially in mathematics. Foundations students also had statistically 
significantly greater gain scores between pretest and posttest than did comparable non- 
Foundations students who were tested under the same conditions and at the same time (see 
Table 6). It is not clear why fourth and especially fifth graders who were not in the program 
gained slightly (but not significantly) more between the pretest and posttest than did 
Foundations students. This anomaly may stem from a selection effect that was not accounted 
for by their pretest scores or perhaps to differences in how well the Terra Nova aligns with 
the Foundations’ curriculum across the different grade levels.” [page 11] 

San Diego’s ‘6 to 6 ’ Extended School Day Program 

San Diego has developed one of the nation’s most ambitious afterschool programs, with the goal 
of making affordable programs before and after school available to every elementary and middle 
school student in the City of San Diego. Two significant evaluations of the program have been 
conducted, one an interim report by WestEd, released in April 2001, the other by Hoffman Clark 
and Associates released in July 2001. Using random sampling of sites, document review, 
interviews, focus groups and site observations, WestEd found: 

> “Parents expressed high levels of satisfaction with the program, “including their perceptions 
of the quality of academic enrichment, the degree to which children looked forward to the 
program, communication with staff, success at helping children complete homework, and the 
promotion of positive behavior in children.” [All bullets from Harvard Family Research 
Project website at http://www.hfrp.org/out-of-school-time/ost-database- 
bibliographv/database/san-diego-s-6-to-6-extended-school-dav-program .1 

> “Almost two-thirds of responding parents noticed improvements in their children’s academic 
performance.” 

Relying on random sampling of program participants and reviewing a variety of data, WestEd’ s 
study concluded: 

> “Reading scores for “6 to 6” students improved. Fifty-seven percent of students increased 
their reading scores over the course of the studied year, and SAT-9 reading scores increased.” 

> “Nearly ten percent of children moved up into the 25 th percentile or higher in 2000 by 
comparison to 1999 reading scores.” 

> “Forty-four percent of students increased their SAT-9 math scores.” 
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The Boys & Girls Clubs’ Project Learn 

Begun in 1996, the Boys & Girls Clubs’ Project Learn focuses on providing youngsters with 
“high-yield learning activities,” including weekly discussions with knowledgeable adults, leisure 
reading, writing activities, homework help, helping others and games that rely on cognitive 
skills. The program has been implemented in full at one-tenth of the Clubs’ 3,300 sites, and all 
sites are implementing components of the program. Steven P. Schinke, Ph.D., of New York’s 
Columbia School of Social Work led an evaluation of the program that relied on a quasi- 
experimental design. Three groups of students were identified, in five separate cities. The 
groups consisted of: 

• Students participating in Boys & Girls Clubs programs that had implemented Project 
Learn, (BGC program sites), 

• In the same cities, students at Boys & Girls Clubs that had not implemented Project Learn 
(BGC comparison sites), and 

• In the same cities, students at non-Boys & Girls Club sites that had not implemented the 
kind of enhanced learning initiatives characteristic of Project Learn (non-BGC 
comparison sites). 

All students in all groups lived in public housing projects, and the sites were chosen to be 
nationally representative of students in public housing. Data on students’ academic performance 
were collected four times: before they began the program, six months after they began, 18 
months after they began, and 30 months after they began. Findings included: 

> “The level of program involvement, as rated by teachers on a scale of 0 to 10, was found to 
be associated with a number of self-reported academic outcomes.” [All bullets from A Profile 
of the Evaluation of the Boys & Girls Clubs of America — Project Learn/Eclucational 
Enhancement Program , Harvard Family Research Project, http://www.hfrp.org/out-of- 
school-time/ost-database-bibliography/database/bovs-girls-clubs-of-america-project-leam- 
educational-enhancement-program . Based on Enhancing the Educational Achievement of At- 
Risk Youth, Steven P. Schinke, Ph.D. et. al.] 

> “As program involvement increased, engagement in reading, use of verbal skills, writing, 
tutoring, and the study of geography all significantly (pc. 05) increased as well.” 

> “Data from teacher reports at final follow-up reveal that program and BGC comparison youth 
more than non-BGC comparison youth had more positive reading skills, writing skills, games 
skills, overall school performance, and interest in class material.” 

> “School grades at 30-month follow-up favored program youth over BGC comparison youth 
and non-BGC comparison youth on overall averages, reading scores, spelling scores, history 
scores, science scores, social studies scores, and attendance. At the 30-month follow-up, 
program and BGC comparison youth had better grades in math than non-BGC comparison 
youth.” 

> “Average grade increases over the 30-month study period were greatest for program youth. 
Program youth increased their average grades by 1 1 percent from baseline to the 30-month 
measurement while BGC comparison youth and non-BGC comparison youth, over the same 
period, increased their average grades by .4 percent and .3 percent, respectively.” 
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> “Program youth missed an average of only 2.19 days of school a year at the 30-month 
measurement as compared to missing an average of 6.4 days a year at baseline. In contrast, 
BGC comparison youth went from missing an average of 4.85 days of school in the baseline 
year to missing an average of 12.33 days a year at the 30-month follow-up. Similarly, non- 
BGC youth went from 7.47 days at baseline to 16.67 at follow-up. The differences between 
the school attendance of program youth and youth in both comparison groups at 30 months 
were statistically significant at p<.05, while they had not been significantly different at 
baseline.” 



* * * * 

The Afterschool Alliance is a nonprofit public awareness and advocacy organization working to ensure 
that all children and youth have access to quality afterschool programs. More information is available at 
www.afterschoolalliance.org. 
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Appendix A 

Afterschool Evaluations at a Glance 



Large-Scale Studies 


Study Subject 


Evaluator/Author 


Release 

Date 


Key Findings 


The Promising After- 
School Programs Study 


Deborah Lowe Vandell, 
University of 
California, Irvine; 
Elizabeth R. Reisner, 
Policy Studies 
Associates, Inc.; and 
Kim M. Pierce, 
University of 
California, Irvine 


October 

2007 


The study of the effect of afterschool programs on nearly 3,000 low-income 
students at 35 high-quality afterschool programs across the nation, found that 
“regular participation in high-quality afterschool programs is linked to 
significant gains in standardized test scores and work habits as well as 
reductions in behavior problems among disadvantaged students.” “Elementary 
school students who regularly attended the high-quality afterschool programs 
(alone or in combination with other activities) across two years demonstrated 
significant gains in standardized math test scores, compared to their peers who 
were routinely unsupervised during afterschool hours.” 


CASEL’s Meta- Analysis 
of the Impact of After- 
School Programs that 
Promote Personal and 
Social Skills 


Roger P. Weissberg, 
President of the 
University of Illinois at 
Chicago-based 
Collaborative for 
Academic, Social, and 
Emotional Learning 
(CASEL) and Professor 
Joseph Durlak, Loyola 
University - Chicago 


January 

2007 


The meta-analysis spanned 73 separate studies of afterschool programs, and 
found that “Youth who participate in after-school programs improve 
significantly in three major areas: feelings and attitudes, indicators of 
behavioral adjustment, and school performance. More specifically, after-school 
programs succeeded in improving youths’ feelings of self-confidence and self- 
esteem, school bonding (positive feelings and attitudes toward school), positive 
social behaviors, school grades and achievement test scores.” 


Mid-continent Research 
for Education and 
Learning (McREL) 2006 
Meta- Analysis of Effects 
of Out-of-School Time 
Programs for At-Risk 
Students 


Mid-continent Research 
for Education and 
Learning (McREL) 
analysis of 35 separate 
evaluations 


Summer 

2006 


The meta-analysis found that afterschool programs had positive and significant 
effects among students at risk of failure in students in reading or math, 
producing positive results on reading achievement, particularly in lower 
elementary grade levels, and in high school students. Researchers also found 
positive and significant effects on math achievement, particularly for middle 
and high school students. 
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Evaluations of State-Level Initiatives and Programs 


Afterschool Learning: A 
Study of Academically 
Focused Afterschool 
Programs in New 
Hampshire 


Susan L. Frankel, Ph.D., 
Kim Streitburger and 
Elizabeth Goldman of 
Portsmouth-based RMC 
Research Corporation 


2005 


The study conducted for the state of New Hampshire of its state- and federally 
funded, academically focused afterschool programs found that “More than half 
the students who attended regularly improved both academically and 
behaviorally.” “Afterschool programs improve students’ learning skills. 
Classroom teachers reported that students made the most progress in turning in 
homework on time, completing homework to teachers’ satisfaction, and 
participating in class.” Further, “Regular attendance seems to contribute to 
student success. ... At the middle school level, where data were richer, more 
students who attended regularly showed academic improvement than students 
who attended less frequently. However, even students who attended 
sporadically showed improvement.” 


Young Scholars Program 
- North Carolina 


Z. Smith Reynolds 
Foundation 


2006 


Young Scholars participants made significant gains on the NC state 
standardized tests in both math and reading. Promotion rates for participants 
increased by 83 percent and regular school day attendance increased. 
Participant grades also improved, with the number of students receiving F’ s 
decreasing by 50 percent while the number receiving A’s and B’s increased by 
38 percent. The study is based on data from 2000-2005. 


Massachusetts After- 
School Research Study 
(MARS) examined 78 
afterschool centers in the 
state 


Intercultural Center for 
Research in Education 
and the National 
Institute on Out-of- 
School Time 


2005 


The MARS study identified program characteristics that are most closely 
related to high quality implementation and explored the links between program 
quality and youth outcomes. Researchers found that staff, especially those 
with a strong educational background, and the presence of a highly qualified 
coordinator or director are key to program quality. 


Texas 21 st Century 
Community Learning 
Centers Programs 


Texas Education 
Agency 


2004 


Strong participation in afterschool correlated with better attendance during the 
regular school day. “Approximately half (48 percent) of youth who 
participated in three quarters or more of the available 21 st CCLC activities 
missed 5 or fewer days of school during the fall semester, compared to 17 
percent of youth who participated in less than one quarter of the available 
activities.” 


California’ s After School 
Education and Safety 
Program (ASLSNPP) 


University of California 
at Irvine working with 
the California 
Department of 
Education 


2002 


A statewide evaluation of California’s After School Education and Safety 
Program (ASLSNPP) demonstrated mathematics gains closely related to 
students’ level of participation in the program: “The scores suggest that, 
particularly for students who participate in the ASLSNPP for substantial 
periods of time, there is a closing of the gap in Math achievement between 
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low-income and other students.” 


The After-School 
Corporation (TASC) - 
NY 


Policy Studies 
Associates 


Five-year 

data 

released in 
2004 
(Reports 
issued each 
year of the 
study) 


Students who participated in TASC afterschool programs attended school more 
regularly and improved their academic performance. Results were especially 
strong for students who participated regularly in the TASC program. At the 
elementary and middle grades level, TASC participants showed gains in math 
achievement and school attendance. At the high school level, afterschool 
participants passed more Regents exams, attended school more regularly and 
earned more high school credits than their non-participating peers. 


Support Our Students - 
NC 


EDSTAR 


2004 and 
2002 


Support Our Students participants’ end of grade achievement test scores 
exceeded the state’s improvement goals and the percentages of students who 
scored at grade level proficiency increased in both math and reading. 
Participants who were the furthest behind and considered the most at-risk made 
the greatest gains. Positive outcomes persist over multiple years. 


Ohio Urban School 
Initiative School Age 
Child Care Project 
(SACC) 


University of Cincinnati 
College of Education 
Evaluation Services 
Center 


1999 


Children in the Ohio Urban School Initiative School Age Child Care Project 
(SACC) had higher test scores than non-participating students statewide. 
“SACC 4 th grade students’ scores exceeded the statewide percentages of 
students meeting proficiency standards in every subject area tested: writing, 
reading, mathematics, citizenship, and science. SACC 6 th graders exceeded the 
statewide percentages of students meeting proficiency standards in four of the 
five areas: writing, reading, mathematics, and citizenship.” 


Local or Program Level Evaluations 


Evaluation of the 
California-based 
Communities Organizing 
Resources to Advance 
Learning initiative 
(CORAL) 


Public/Private 
Ventures’ Amy 
Arbreton, Jessica 
Sheldon, Molly 
Bradshaw, Julie 
Goldsmith with Linda 
Jucovy and Sarah 
Pepper 


February 

2008 


A Public/Private Ventures evaluation of the “Communities Organizing 
Resources to Advance Learning” (CORAL) project, an eight-year $58 million 
afterschool initiative of the James Irvine Foundation, concluded that the five- 
city program’ s tightly focused literacy programming three to four days a week 
produced “pronounced gains in achievement for a range of students.” The 
programs were all based in California. Most afterschool students were 
elementary-school aged. More than half were designated English learners, and 
89 percent were recipients of free or reduced-price lunch. The CORAL 
programs are in Fresno, Long Beach, Pasadena, Sacramento and San Jose. 


Chapin Hall’s After- 
School Programs and 
Academic Impact: A 
Study of Chicago’s After 


Chapin Hall Center for 
Children at the 
University of Chicago 


January 

2007 


Students who participated in the program missed fewer days of school than 
their classmates, and those who participated most frequently failed fewer core 
academic courses (English, Math, Science and Social Studies). The finding is 
particularly noteworthy because the program is not aimed at improving 
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School Matters 






academics. By creating an incentive for students to attend school regularly and 
giving them something to look forward to after school, researchers conclude, 
After School Matters helped improve academic performance. Also, over their 
full high school careers, students enrolled in the program for three or more 
semesters and those who participated at the highest levels had higher 
graduation rates and lower dropout rates than similar students not in the 
program. 


Citizen Schools - sites in 
MA 


Policy Studies 
Associates 


January 
2008 (most 
recent 
installment 
of ongoing 
five phase 
evaluation 
report) 


Former 8 th Grade Academy participants were more likely to participate 
and enroll in high-quality high schools and improve attendance than 
matched nonparticipants. Former participants with high levels of 
program exposure had a significantly higher rate of on-time promotion 
to tenth grade than did matched nonparticipants. Overall, former Citizen 
School participant were more likely to pass the tenth-grade Mathematics 
and English/Language Arts MCAS tests than were students district- 
wide. 


YMCA of Greater New 
York’s Virtual Y 
Program 


National Center for 
Schools and 
Communities at 
Fordham University 


2005 


Virtual Y students outperformed a comparison group in attendance and 
mathematics. Findings are based on seven years of research. 


Generacion Diez 
Adams County, PA 


Prevention Research 
Center at Pennsylvania 
State University 


Data 
collected 
between 
2001 and 
2003 


Generacion Diez participants’ test scores in reading, spelling and math 
increased significantly. Students demonstrating greatest gains were from 
families that were less in engaged in school and/or functioning at a lower level. 


Mahoney and Lord 
Ecological Analysis of 
After-School Program 
Participation - examining 
students at three 
unnamed public schools 
in the Northeast 


Joseph Mahoney and 
Heather Lord, Yale 
University and Erica 
Carry 1 of New York 
University 


July 2005 
(first 

installment 
of ongoing 
evaluation) 


In a study comparing afterschool students with those cared for by parents, by a 
combination of siblings or in self care, and by a combination of other adults 
and in sibling or self care researchers found that children who were in the 
highest category of afterschool program attendance had significantly higher 
reading achievement than children in all other care arrangements. 


LA’s BEST 
Los Angeles, CA 


UCLA Center for the 
Study of Evaluation 


2006 (most 
recent 
release of 


The newest report issued from the study found that LA’s BEST participants 
were 20 percent less likely to drop out of school than non-participants. 
Previous reports in the evaluation show that students’ regular school-day 
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decade-long 
study with 
series of 
reports 
issued) 


attendance improved once they began participating in the afterschool program. 
That led to higher academic achievement on standardized tests of math, 
reading and language arts. In addition, language redesignation rates favored 
LA’s BEST students when compared with non-LA’s BEST students. 


Pathways to Progress - 
St. Paul, MN 


Center for Applied 
Research and 
Educational 
Improvement, 
University of Minnesota 


March 2004 


Pathways participants outperformed comparison students on standardized tests 
in reading and math, received better grades in English and math and attended 
school more regularly. The study is based on data collected between 2000 and 
2003. 


Foundations, Inc. - 
PA and NJ 


Gansk & Associates 


2003 (most 
recent 
release in 
series) 


Foundations, Inc. students at every grade level scored higher on math, reading, 
and language arts standardized tests than the comparison group. The study is 
based on data collected from 2001-2003. 


San Diego’s “6 to 6” 
Extended School Day 
Program - 
San Diego, CA 


Two reports; one by 
WestEd and one by 
Hoffman Clark and 
Associates 


2001 


WestEd reported that reading scores for San Diego’s “6 to 6” students 
improved. Fifty-seven percent of students increased their reading scores over 
the course of the studied year, and SAT-9 reading scores increased. Nearly ten 
percent of children moved up into the 25 th percentile or higher in 2000 by 
comparison to 1999 reading scores. Forty-four percent of students increased 
their SAT-9 math scores. 


The Boys & Girls Clubs’ 
Project Learn 


Columbia School of 
Social Work 


2000 


A five-site evaluation of Project Learn over 30 months found that engagement 
in reading, use of verbal skills, writing, tutoring, and the study of geography all 
significantly increased as program involvement increased. Program youth 
increased their average grades by 1 1 percent from baseline to the 30-month 
measurement while comparison youth increased their average grades by less 
than 1 percent. 
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